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ABSTRACT 

For an evaluation design or ''political strategy" to 
succeed in instituting a process of valuing and be appropriate for a 
particular setting, patterns of social relations and beliefs about 
the evaluation must be in agreement with those of the chosen 
evaluation design. This agreement must be in place either at the 
onset or occur during the evaluation process. Alignment of political 
power and definitions of credible knowledge is inherent in six 
evaluation approaches or designed which include: positivism, 
interpretivism, critical theory, aesthetics, collaborative research 
and action research. The sociology of knowledge approach is used to 
examine the social meaning of each of these designs. This analysis 
exposes much of the taken for granted assumptions in each of these. 
Specifying types of evaluation designs that are appropriate is to 
define a specific potential role benefiting specific parties in the 
evaluation and in the educational system itself. Educators and 
evaluators should consider the need for, the design of and political 
outcomes likely in any evaluation situation and to eschew general 
policies about the necessity and form of evaluations. Designs should 
be examined from a larger perspective to understand the full social 
meaning of educational evaluation. (RR) 
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INTRODUCTION 

Evaluation research continues to flourish as issues of accountability 
and effectiveness gain attention in education. As evaluation research increases 
in necessity and popularity, so do claims of the "best" or "most appropriate" 
evaluation approaches or designs* Presumably, the results of such research are 
used to justify or change educational policy* House (1984:184) makes the case 
that each "evaluation approach entails a significant role for evaluation within 
the decision process"* He claims that, in fact, some of the differences in 
designs can be accounted for by the anticipation of a different decision 
situation inherent In the design* Thus, a choic^ of an evaluation design is, in 
essence, a policy decision* 

*Paper prepared for American Educational Research Association annual meetings, 
April 20 - 24, 1987, Washington, D.C. 
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Like other educational policies, educational evaluation can be 
subjected to analyses; this is rarely done. House (1980) argues that, to 
understand evaluation approaches, it is necessary to understand the ethics, 
epistemology and political ramifications of each. When such an attempt at 
analyses and understanding does occur, we find evaluation has the earmarks of a 
political strategy. Bredo and Feinberg (1982:430) suggest that an orientation 
towards research "implicitly represents a different stance towards existing 
authorities or a different political position." House (1980:78) insists that 
there must be agreement between fill parties— the sponsor, the evaluator, and the 
evaluatees~as to "criteria, methods and procedures, access, dissemination of 
results and so on" for "disagreement . . . can destroy the' entire credibility of 
the evaluation." Noblit (1984: 96) argues that any sort of applied research is 
"inherently political because it wishes to establish the bases of judgement for 
others and moreover to replace "hose that might otherwise be employed." The 
choice of the evaluation design, then, is more than a technical issue. Such 
designs realign political power aid redefine what is credible knowledge. 

In this study we will examine the alignment of political power and 
definitions of credible knowledge inherent in six evaluation approaches or 
designs: positivism, interpretivism, critical theory, aesthetics, collaborative 
research and action research* We will make the case that the issues of 
political power aiid credible knowledge take different forme In each of these 
evaluation designs. The power and credibility of those in charge (or the 
sponsor), the evaluator-researcber, the evaluatees and even the evaluation 
design itself are all at issue. Yet, beneath these more obvious issues of power 
and credibility are the more subtle politics of social network (Schmidt, et al.» 
1977) and the relative credibilities of the various knowledge bases in the 
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evaluation situation^ We assert that each evaluation design implicitly promotes 
patterns of social relations and particular knowledge bases and assumptions. 
Thus, we argue that for an evaluation design or "political strategy" to succeed 
in instituting a process of valuing and be appropriate for a particular setting, 
patterns of social relations and beliefs about the evaluation must be in 
agreement with those of the chosen evaluation design; this agreement must be in 
place either at the outset or occur during the process of evaluation. This 
agreement, in the form of a dominant coalition (Benson, 1975), is an essential 
part of the political strategy of educational evaluation. Finally, we conclude 
that all the major approaches to evaluation are essentially political strategies 
used to create a dominant coalition. 

The examination of evaluation designs as political strategies is 
appropriately a sociology of knowledge problem. The sociology of knowledge 
concerns itself with social bases, construction and effects of forms of 
knowledge such as evaluations (Berger and Luckman, 1966). In this case, we will 
use the sociology of knowledge approach to examine the social meaning of each of 
the six evaluation designs. First, we will describe each design as it is used 
in evaluation, examine the implicit or explicit patterns of social relations 
using Schmidt jBit al. 's (1977) grounded analyses of patron-client and horizontal 
networks, and determine which beliefs and knowledge bases are primary in each 
design. Second, we will propose some ideas about the political strategy of 
evaluation design using Benson's categories of strategies employed to bring 
about change in social networks (Benson, 1975). Finally, we will make some 
preliminary suggestions as to the appropriateness of these political stratej^ies 
under specific evaluation conditions. 



A SOCIOLOGY OF KNOWLEDGE APPROACH 

In order to consider evaluation designs as political strategies, it 

is necessary to understand the sociology of knowledge approach we will use to 

examine the social relations and belief systems that characterize the specific 

evaluation design. As Berger and Luckraann (1966; 3) write; 

It is our contention, then, that the sociology of knowledge roust 
concern itself with whatever passes for "knowledge" in a society 
regardless of the ultimate validity or invalidity (by whatever 
criteria) of such "knowledge". And insofar as all human knowledge is 
developed, transmitted and maintained in social situations, the 
sociology of knowledge must seek to understand the processes by which 
this is done in such a way that a taken-for-granted "reality" congeals 
for the man in the street. 

This approach has been employed with some success in the study of 

education. Noblit (198A) used this approach to examine the prospects for 

ethnography in educational research and evaluation, revealing that the tenets of 

qualitative research are threatened in applied research endeavors. Bowers 

(198A:vii) went beyond an analysis of relating effects of modernization "to show 

how the sociology of knowledge can be used to develop a theory of education." 

In this article, we will examine evaluation designs as proposed sets of social 

relations and beliefs that evaluators wish to establish in evaluation 

situations. Our argument is that evaluators seek to create a set of social 

relations and beliefs that facilitate access to the evaluation situation, 

develop commitment of participants to the evaluation, and enable the evaluation 

to be politically salient. In this way, we argue evaluation designs can be 

considered, at least in part, as political strategies. 

To examine the social relations of evaluation design, we will use 

social network theory (Schuixuu, et al , 1977). The study of social networks has 

developed as a grounded "theory" of social relations that has been termed 



"political clienteliso" (Schmidt, et al , 1977)* In its briefest form, political 
cllentellso posits that networks are maintained by exchanges of favors in such a 
way that an obligation to reciprocate is engendered* Some (horizontal) networks 
can be more or less of equals in power and status, and network relations 
maintain that equality* Other (vertical) networks consist of patrons and 
clients* In these networks, clients typically exchange deference and loyalty to 
the patron for the patron's protection and support. Again, network relations 
maintain this essential inequality* Evaluation situations generally Involve 
three sets of parties: the evaluators, the evaluatees, and the sponsors of the 
evaluation, although in some designs the sponsors may well be the evaluatees* 
We will use political clientelism to analyze the sets of social relations and 
relative power among these parties in each design* 

Belief systems may be conceptualized in many ways — as culture, 
values, and/or ideology. Yet in evaluation, the essential beliefs concern the 
relative credibility of the knowledge systems of the parties to the evaluation* 
Evaluation designs are intended to establish the credibility of the knowledge 
the evaluation generates (House, 1980)* For our purposes, it is necessary to 
examine the relative credibility of the parties* knowledge systems in each 
design and how designs may seek to alter this in the process of creating a 
credible evaluation* 

In what follows, we will reconsider the evaluation designs of 
positivism, interpretivism, critical theory, aesthetics, collaborative research 
and action research as political strategies* The six designs are not always 
distinct* Collaborative researchers may employ interpretivist ideas, as may 
aesthetics* Action researchers may be rather positivistic* Critical theory is 
maintained to encompass and go beyond both interpretivism and positivism* 



Research projects may be both collaborative and action research. Yet since each 
evaluation design seeks to create a distinct set of social relations and 
beliefs, it is useful to cxatcine the six approaches as discrete entities. 
Farther, in the sociology of knowledge, all six approaches are worthy of 
parallel consideration. Some may wish to accord a lofty status to positivism, 
interpretivism and critical theory. But, following Berger and Luckmann, we 
would caution such a pre'-umption. All are ways of knowing in evaluation 
research and deserve parallel treatment. Because all are parallel for the 
purposes of this analysis, the terms "designs" and "approaches" are used 
interchangeably and do not connote a hierarchical ordering. 

This reconsideration of evaluation should not be taken to imply that 
we believe evaluation designs are only political strategies, for that is not the 
case. Evaluation designs are mcny things besides political strategies, but we 
would argue at their base they are political strategies. Further, ^t should not 
be inferred that because we see evaluations as political that evaluations are 
not worthy social processes. If anything, the analysis that follows establishes 
that evaluations are recognizable processes through which values and, thus 
worthiness, is created. 

EVALUATION DESIGNS 
Positivism 

Positivism extolls science as the superior way of knowing and the 
scientist as the expert, or credible agent. Knowledge is discovered through a 
reductionistic episteraology using a traditional scientific methodology. 
Objectivity is a key element in this methodology and, according to Kerlinger, 
"is a most important methodological aspect of science" which Is achieved when 
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"the procedure can be repeated with the same or similar results" (Kerlinger, 
1979. 8-9) • Furthermore, l:e states that "objectivity is important because it can 
help provide more trustworthy explanations of natural phenomena" (Kerlinger, 
1979: U). In this approach, human systems are viewed in terms of a linear 
cause-effect model. Human events are seen as part of the natural world and, 
therefore, lawful. Smith argues that "these laws describe in neutral scientific 
language how . . . independently existing reality really operates" (1983:11). 
He continues by saying that the laws are, "by definition, universally applicable 
regardless of time and place" (Smith, 1983:11), As applied to educational 
evaluation, a positivistic design provides a utilitarian approach to solving 
evaluation issues "to explain, and by extension to be able to predict, the 
relationship between or the invariant succession of educational objects and 
events" (Smith, 1983:11). 

The evaluator-seientict "role in this approach is one of the credible 
expert. His/her scientific expertise is used to legitimate this status as well 
as the evaluation Aesign itself. Interpersonal skills are only minimally 
required and the relationship between the evaluator and evaluatees is often 
distant in the pursuit of objectivity. The evaluator-scientist relies on the 
authoritarian relations between the sponsor and the evaluatees to gain access to 
and maintain relations with the evaluatees. 

The implementation of a positivistic evaluation design clearly 
assumes social relations which are defined according to carefully delimited 
patron-client networks. Initially, the evaluator-scientist is client to the 
patron-sponsor. He/she must show deference and loyalty during the careful 
negotiation of contract domains to gain access to the evaluatees and to assure 
power and credibility during the evaluation process. At this juncture, the 
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evaluator-scientist theoretically becomes patron to both the sponsor and 
evaluatees* However, the sponsor aay maintain certain aspects of patronage 
toward the evaluator as regards evaluation results upon which sponsor decisions 
are to be made» After access is gained, the evaluatees become a clientele to 
the evaluator-patron» This patron status is largely based in technical 
expertise and is reinforced by the access given to the evaluator by the 
sponsor* 

The sponsor in this design is, in the end, also a client to the 
patron-3cientist while continuing to be a patron to the evaluatees. If weak 
horizontal networks »re present among the evaluatees, as might be expected in 
loosely-coupled educational organizations (Weick, 1982), evaluatee clientelism 
is maintained by authoritarian relations previously established between the 
evaluatees and sponsor. However, if strong horizontal evaluatee networks exist 
or develop during the evaluation process, and these networks have an on-going 
basis of exchange, the evPxuatee client networks can gain power and undermine 
the evaluation by playing off one patron against the other (i*e., the sponsor 
and the evaluator). 

The belief system required by the positivistic design is one of 
reification of science. Credibility rests on the agreement by all parties that 
science is indeed an appropriate basis upon which to evaluate and make 
decisions. The utilitarian nature inherent in this model would seem to demand 
that knowledge gained would be instrumental or "practical", presumably for the 
evaluatees. Hcvever, such knowledge may be Instrumental only in terms of 
assuring a justification for sponsor decisions and reinforcement of science and 
the scientific method as appropriate (House, 1980). Thus, knowledge discovered 
through a positivistic evaluation model is in service of patrons rather than 
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clients with the design serving to promote the authority of legitimate rather 
than authentic knowledge (Gouldner, 1070). 



Interpretlvism 

Interpretivism, in many ways, is the original basis of sociology and 
anthropology and recently has been experiencing a considerable resurgence^ As 
Patten (1980) has argued, it is a dramatic alternative to positivism, focusing 
on putting the meaning of social situations in relevant contexts (Mishler, 
1979). Spicer (1976) proposes that the approach is characterized as 1) emic 
(grounded in the experiences of the participants); 2) holistic; 3) historical; 
and 4) comparative. In interpretive evaluations, the evaluator typically 
observes and interviews the parties to the evaluation to construct a "reading" 
(Geertz, 1973) of the "multiperspectival realities" (Douglas, 1976) of the 
situation being evaluated. The result typically is a written account taking the 
form of an ethnography or case study that, at least in part, is submitted to be 
reviewed by the various parties to the evaluation. While interpretivists often 
view their role to end with the completion of the research endeavor and its 
sharing, they will often propose that taking action based on the evaluation is 
not as straightforward as the sponsor or evaluatees may believe « As 
interpretivists will argue, the source of problems in an educational program may 
largely be the assumptions involved in crea*:ing the situation, and less so in 
technical deflciences in program design or implementation. As evaluations, 
interpretivist accounts often have low i^iStrumental value and may yield 
unanticipated results and other consequences. 

Interpretivists see their evaluator role to be one of revealing such 
taken-for-granted assumptions. Since interpretivists are cautious about 
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proposing an Instrumental value to their evaluations, they focus more on 
developing relationships that first provide access to the situation and, over 
time, create trust in the evaluator. For interpretivists, though, the question 
of "whose side are we on?" (Becker, 1967) looms throughout the evaluation^ This 
question is resolved either by consciously "going native" (Wolcott, 1977) or by 
providing a descriptive account that puts the case of each "side" into an 
understandable context. Thus, the evaluator inust have considerable experience 
in data collection techniques and in rendering a coherent and credible account 
(House, 1980) • 

The social relations of an interpretive evaluation, thus, are complex 
and changing. The interpretivist usually negotiates for the unique status of a 
"voyeur", a person who is able to watch universally but reserves the right to 
decide when participation is appropriate. In doing so, the interpretive 
evaluator maintains u distance from the normal authority structures present in 
the situation, carefully avoiding becoming an exclusive member of any network. 
This role, then, is of a "broker" between social networks, yet withholds 
actually transmitting messages or facilitating social exchanges until the end of 
the evaluation. 

The evaluatees in an interpretive evaluatica are expected to grant 
access of various sorts to the evaluator. However, they are not assumed to 
trust the evaluator or the evaluation, as interpretivists view trust as being 
earned through ongoing social interaction. In granting access, the 
sponsor-evaluatees in many ways are behaving altruistically. Acc ss, then, is 
the result of a sponsor with sufficient authority and/or patron status vis-a-vis 
the evaluatee*s clientelism, or of existing inter-network relations that include 
the evaluator prior to the evaluation. In any case, the evaluator avoids being 

^ 1 t 
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either a patron or client, while the evaluatee becomes dependent jn the good 
faith of the evaluator. In cur experience, both enoisors and evaluatees resolve 
this by coming to believe that the evaluator s account will vindicate their 
position and actions (cf* Collins and Noblit, 1978)* 

Unlike some of the other evaluation designs discussed hers, 
interpretivism does not entail a prior belief concerning the legitimacy of the 
approach; rather only access is required, Interpretivists disavow the usual 
bases of legitimacy such as content expertise, instrumental utility, or 
authority. Yet it is expected that over time both trust and the legitimacy of 
the evaluation will be negotiated* That is to scy, interpretivists seek to 
create a belief in the legitimacy of the evaluation in question, and 
interpretivism in general. In this way, interpretivists seek converts, and if 
the conversion is complete, assume the role of a compassionate and even-handed 
patron to the evaluatees and sponsor. Loyalty and deference may be exchanged 
for the protection of a "democratic" multiperspectival reality and the support 
of the evaluator^pa :on. If the interpretivist-evaluator cannot achieve this 
patron status, the evaluation may be rejected as biased «nd/or not useful. In 
the end, interpretivist evaluations are constructivist of both belief systems 
and patron-client relations. 

Critical Theory 

Critical theory as a mode of evaluation is not as popular as the other 
approaches we discuss here (Bredo and Feinburg, 1982). Yet we have seen its 
popularity increase in education in recent years (cf. Giroux, 1981) • Critical 
theory is essentially the critique of ideologies which justify domination. In 
Habermas' formulation, ideology distorts communication by masking social 
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contradictions, making it difficult for individuals to discern the ideological 
content of the beliefs that structure their lives and their consciousness 
(Geusss 1981) • The critical theorist would have people emancipated from 
ideological domination 'through a program of dialogue and discourse (dialogue 
about the nature of communication itself) designed to promote self-reflection 
and, consequently, enlightenment and emancipation* Such an evaluation program 
requires, however, that the evaluator can create an approximation of an "ideal 
speech situation" which allows free and uncoerced discussion (Habermas, 1970) • 
Comstock (1982:379-386) details the critical method as involving seven steps: 
1* Identify movements or social groups whose interests are 
progressive. 

2. Develop an interpretive understanding of the intersubjective 
meanings, values, and motives held by all groups of actors in the 
subjects' milieu. 

3. Study the historical development of the social conditions and the 
current social structures that constrain the participants' actions 
and shape their understandings* 

A* Construct models of the determinate relations between social 

conditions, intersubjective interpretations o^ those conditions, 

and participants* actions* 
5* Elucidate the fundamental contradictions which are developing as a 

result of current actions based on ideologically frozen 

understandings* 

6* Participate in a program of education with the subjects that gives 
them ne'-' ways of seeing their situation* 
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?• Participate in a theoretically grounded program of action which 
will change social conditions and, in addition, will engender new 
less alientated understandings and needs. 

An evaluation based in critical theory is largely participative ^ 
trying to facilitate the evaluatees* free and uncoerced discussion of their 
situation* Yet if we examine Comstock's program, we see that the role of the 
evaluator-critical theorist is one of first among equals. The 
evaluator-critical theorist has expertise in critique as a genre as well as in 
the process of facilitating the ideal speech situation and the dialogue and 
discourse that ensues. Further, to the extent that such discourse must be based 
in evidence about social conditions, intersubjective meanings and the 
connections between the two, the evaluator-critical theorist also may have roles 
approximating those of the positivist or interpretivist. 

The social relations involved in a critical theory evaluation are 
obviously delicate. The evaluator-critical theorist is in many ways a supreme 
patron, providing both content and process direction in the service of free and 
uncoerced dialogue. The evaluator-critical theorist must continually legitimate 
critique as a g^nre and reassure the evaluatees of the value of shedding their 
false consciousness. The evaluatees, at least initially, are clientele to the 
critical theorist's patronage, in the sense that they must commit to follow the 
patron to some enticing, yet unspecified, and in many ways, unpredictable end. 
They engage in the approximation of the ideal speech situation as equals among 
themsplves, free to discern ideological distortions as they see them, and free 
even to decide not to proceed with a course of action once ideologies are 
revealed (Geuss, 1981). Yet they are not equal to the evaluator-critical 
theorist in creating the content and process through which this occurs. An 
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evaluation based In critical theory would be wary of an outside sponsor, since 
the interests of a sponsor may well perpetuate ideological distortions* Any 
outside sponsor would, of necessity then, be more of a philanthropist, providing 
resources for others to do with as they wish* 

As is evident in Comstock's program, a critical theory evaluation 
requires a set of prior beliefs* This prior set of beliefs is what Collins 
(1982) refers to as the Durkheimian notion of the "precontractual basis of 
solidarity" inherent in social contracts* Durkheim has posited that every 
social contract, in this case that of evaluation, actually entails two 
contracts* The first is the consciously agreed-upon contract, that of engagii 
in an evaluation using a particular approach* The second is the "hidden 
contract" that rests on the implicit assumption that all participants agree to 
the rules of the first contract (Collins, 1982)* Thus, a critical theory 
evaluation, like the remainder of the designs we will examine, entails this 
precontractual basis of solidarity or set of prior beliefs* 

In critical theory, the agreed-upon contract includes agreement 
concerning the need to shed delusions, a predisposition to critique as the genre 
in which to do so, and an interest in emancipation* Beliefs in the instrumental 
or technical value of the evaluation is not required and indeed may be exposed 
as ideology in the process* What is intriguing about the critical theory 
approach is that while the evaluator is a strong patron, there is no suspension 
of belief in the vested interests of the evaluator-critical theorist* Indeed, 
since to critical theorists all knowledge has interests (Habermas, 1971), the 
interests of the evaluator, as well as of evaluatees, are subject to examination 
and reflection* Nevertheless, as part of the "hidden contract," a critical 
theory evaluation in the end requires that the evaluatees believe that the 
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evaluator is serving their best interests* 
Aesthetics 

Aesthetics is another type of qualitative approach to evaluation 
(Eisner, 1979; House, 1980). In Eisner's formulation, aesthetic evaluation 
Involves both "connoisseurship" and "criticism". Connoisseurship involves the 
"art of appreciation" (p. lA). The evaluator must have "developed a highly 
differentiated array of anticipatory schema that enable one to discern qualities 
and relationships that others, less well differentiated, are less likely to see" 
(p» 14)* Connoisseurship is necessary to aesthetic evaluations In that "... 
it provides the content for our knowing. It makes possible the stuff we use for 
reflection" (p. 15). Yet to Eisner, connoisseurship is private, not public. To 
make it public, it must be transformed into a form that others can understand. 
This is the role of criticism. Criticism entails first creating an artistic 
description so that others may "vicariously participate" (p. 15) in the events 
at issue. Second, criticism includes rendering an interpretation by ". . . 
applying theoretical ideas to explain the conditions that have been described" 
(p. 16). Third, criticism involves an appraisal. This appraisal is not in the 
form of an outcome--based evaluation. Rather it is to provide constructive 
criticism, "... providing the conditions that lead to the improvement of the 
educational process" (p. 16). 

Eisner and his students have conceived of the role of the evaluator to 
be one that provides "a fresh eye" (p. 17). To do so, he aigues that 
interpersonal skills and trust are essential between the critic and evaluatee: 
"The teacher must be willing to have a critic in the classroom and roust be 
willing to listen (but not necessarily heed) to what the critic says" (p. 17). 

16 



16 

In Eisner* 8 vlew» this relationship is one of a dialogue between friends* 

Yet on closer analysis, the social basis of aer>thetlc8 involves social 
relations that are not typically friendship relations^ The evalua tor-critic 
must hsve considerable expertise to be recognized as credible » yet also be 
sufficiently independent of other power and authority relations so that the 
evaluatee is willing to participate in good faith and to trust the 
evaluator-critic» The evaluatee must believe that improvement is so desirable 
as to seek a "fresh eye." Nevertheless, the evaluatee is dependent on the 
critic for insight and direction^ Eisner argues that schools should provide 
"structures" (p#17) for observation and reflection, but is clear that 
connoisseurship and criticism concerns the particular not the universal, 
rendering aesthetic evaluation not amenable to bureaucratic ends* 

The social relations between evalua tor-critic and evaluatee involve a 
subtle dependency of the evaluatee on the expertise of the evaluator-critic 
without requiring compliance, much like the ideal speech situation in critical 
theory* Any sponsors outside of this dyadic relationship are s.imply to provide 
the opportunity for the evaluation, with the faith that improvement will occur* 
Intriguingly , the evalua tor-critic is not in a patron status as he/she is unable 
to provide protection and support within the authority of the educational 
organization* The sponsor, on the other hand, is more like a patron of the arts 
than a direct supervisor tc either party* He/she must believe in the value of 
aesthetics and sponsor its practice, but not be assured of any instrumental gain 
for the organization, except possibly in social status or in the evaluatee "s 
internal motivations* 

Aesthetics, like other designs, seem to involve a precontractual 
basis of solidarity (Collins, 1982)* The conscious contract is an expression of 
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belief in the legitimacy of art criticism as applied to educational practice^ 
Parties must believe that the pursuit of creative expression is instrumentally 
valuable. On the other hand, the hidden contract requires a suspension of 
belief on the part of the evaluatee in the vested interests of the 
evalua tor-critic and sponsor. Altruism for all parties is assumed. Trust and 
skills in interpersonal relations are to hold the relationships and beliefs 
together. Aesthetic evaluations require that the evaluation be not a j;oal but 
m expression of key values that in the end reinforce an altruistic belief in 
aesthetics. 

Collaborati^^e Research 

Torbet (1981:150) defines collaborative inquiry as "self-study in 
action"— an incomplete, ongoing, experiential process. Collaborative research 
assumes that research and action are inseparable, except in an analytic sense, 
and that knowledge comes through and for action. Collaborative inquiry 
diminishes some of the substantive differences that can be present among 
practitioners, sponsors and evaluators (Schlechty and Noblit, 1982) since the 
"we-ness" of research is emphasized in this approach. Tn collaborative 
evaluation, all aspects are negotiated—the research design, the roles of all 
participants and the issues. The design of collaborative inquiry, then, is not 
pre-defined nor necessarily stable but is an evolutionary, developmental process 
(Torbet, 1981). 

The evaluator in collaborative inquiry must develop a "shared 
reality" with all other participants in terms of belief in the collaborative 
process, role domains and evaluation issues. Thus, the evaluator's role must be 
or must becc!:e one of an interested agent within the evaluation process. 
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Collaborative inquiry in its "purest" form requires relative equality 
of power* This requirement presupposes that social relations are in place prior 
to the evaluation and that socially enforced equality is maintained through 
negotiation and active bargaining. Networks within a collaborative design are, 
in essence, an alliance based on mutual trust and belief rather than one of 
patrons or clients. Thus, whether the participants are technically evaluator, 
sponsor or evaluatee, they must operate as a horizontal network with the 
interests of all parties given equal consideration. Yet if collaboration is to 
achieve an evaluation or a reevaluation of a setting, it must avoid "group 
think" characteristics of groups together over time. This typically is the job 
of the evaluator (Schlechty and Noblit, 1982; Newman and Noblit, 1982) who takes 
the role of representative. The evaluator in the collaborative endeavor 
represents perspectives from outside the evaluation situation as well as 
representing the collaboration's perspectives to wider audiences. The evaluator 
thus assumes an instrumental expertise as a translator. In social networks, 
this is akin to the role of a "broker" who, in transmitting a message, also 
invariably alters its content (Lande, 1977). Collaborative research creates a 
horizontal network pxid ideally avoids creating patrons. Yet the 
evaluator-as-broker is a boundary-spanning member of the social network and thus 
has a subtle, manipulative power upon which the fruits of collaboration are 
dependent. 

It is also true that evaluations using a collaborative design require 
a "precontractual basi. of solidarity" (Collins, 1982). Legitimate knowledge is 
process knowledge, not substantive knowledge. The reification of this 
collaborative process is the basis of the conscious contract. In practice, 
however, the hidden contract requires a prior trust in the other participants 
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that the negotiated social contract will not be violated. This social contract 
is definitive only in requiring collaboration and not in specifying a 
substantive knowledge base as credible* 

Essentially, there are no real "results" in a collaborative 
evaluation as these evaluations can be considered ongoing "experiments in 
practice" (Torbet, 1981:1A7)# Certainly there is no seeking after instrumental 
or legitimate knowledge. Rather, Torbe*: (1981:151) states that collaborative 
inquiry is a seeking of "valid social knowledge" for the participants to 
develop and apply to their everyday lives* 

Action Research 

Action research has sz'*-'' 1 commonalities with collaborative research 
and, in fact, differences in the two may be virtually non-discernable in actual 
practice* However, we believe theoretically there are differences worth 
examining in the context of evaluation as political strategy* 

Action research insists that the interests of the practitioner be 
primary* The particular method of the evaluation is not as important as its 
appropriateness to the environment, problem and participants (Nixon, 1981)* 
Although there is variability in action research designs, Sanford (1981:178-'9) 
promotes a model of action research entailing the following features: 1) 
practical, open^-ended questions, 2) the promotion of individual development, and 
3) practitioner planning which supersedes planning by "experts"* Because of its 
emphasis on practicality, appropriateness and applicability, action research may 
actually be antithetical to evaluation as a process of revaluing a situation* 
Nonetheless, the action research approach is being adopted as a mode of 
evaluation and supervision in many situations, perhaps bec*^use it attempts to 
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fill the gap between research and practice through a practice emphasis* In 
fact 9 action research uses evaluatee or practitioner world view as the most 
credible knowledge base* 

The role of the evaluator within action research requires that he/she 
suspend all personal and professional beliefs about the evaluation issue(s) and 
setting and believe single-mindedly in the priority of the practitioner* The 
evaluator must establish him/herself as a credible technician to the 
practitioner as well as a trusted reporter at the conclusion of the evaluation* 

This evaluation design is the only one in which the evaluator is 
ultimately the client to the evaluatees* Although this evaluator clientelism 
changes somewhat from the initiation to the conclusion of the process, the 
social networks remain relatively intact* Initially, the evaluator evidences 
deference and loyalty to the patron-evaluatees through his/her total attention 
to practitioner-defined Issues* The evaluator may prompt action, but it must be 
justified in terms of practical knowledge as defined by the participants* At 
the conclusion of the process, the evaluator assumes a representative role, 
being charged with reporting the results obtained* However, any evaluatioi: 
results would have been previously approved by the evaluatees (Sanford, 
1981:178), reinforcing their patron status* Evaluatees would not necessarily 
feel the need to fulfill a clientele role of deference and loyalty to the 
evaluator since sponsor and evaluator power and authority do not exist withXn 
this design* Should a sponsor have a role in the process, generally in the 
initiation phases, the sponsor essentially ^Misappears** as is the case in the 
aesthetic design* 

The belief system inherent in action research again requires a 
"precontractual ba^is of solidarity" (Collins, 1982)* Action research is 
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predicated on the prior agreed-upon contract that the practitioner is the expert 
and that the process and .results of action research are legitimate and credible. 
Practitioner knowledge, or local knowledge (Geertz, 1973) is tthe significant 
knowledge base. The knowledge gained is considered authentic end instrumental, 
unlike the more formal knowledge bases ci positivism, interpretivism and 
critical theory. Like the aesthetic designs, the hidden contract in action 
research necessitates the suspension of belief by the evaluatees concerning the 
vested interests of the sponsor(s) and/or the evaluator. Any sponsor operates 
under the altruistic belief that tht process and results will be valuable for 
the practf.tioner-evaluatees and, therefore, that the action research evaluation 
is justified. Moreover, action research is as technical as positivism for it 
reifies practical and authentic knowledge. However, unlike positivism, it does 
so in the service of the interests of the usual underdog in evaluations, the 
practitioner. 



THE POLITICAL STRATEGY OF EVALUATION 

We have made the case in this paper that evaluation is a 
socially-created "reality" that alters social relations and beliefs during the 
process. Because of these alterations, we hcve posited that evaluation designs 
are actually political 8tra::egies and that evaluation as a successful political 
strategy seeks to maintain or create a dominant coalition (Benson, 1975), Each 
of the evaluation designs we have discussed seek dominant coalitions of some 
sort. The designs vary in the political basis of r.he coalition and the 
potential political outcomes. 

Ben'-on (1975) defines four strategies employed to produce social 
network change and attain the social agreement necessary for the formation of a 
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dominant coalition. These four strategies — authoritarian, manipulative, 
cooperative, and disruptive — are predicated upon conditions of relative power 
imbalance or balance* Benson argues that authoritarian and/or disruptive 
strategies are necessary for successful network change under conditions of 
substantial power imbalance. Manipulative and cooperative strategies must be 
employed where relative autonomy or power vis-a-vis all other parties exists 
(Benson, 1975). Using Benson's conditions for strategic network change in 
combination with our examination of social network relations, it becomes 
apparent that each of the six designs reflect patterns of relative network 
power. Strong patron-client networks, and thus a power imbalance, exist within 
three designs — positivism, critical theory and action research. Relative 
autonomy or power equality forms the basis for the remaining three — aesthetics, 
interpretivism and collaboration. 

Benson discusses resources as the other significant factor in his 
discussion of political strategies for change. We would assert that beliefs 
about credible knowledge are the major resource in the evaluation situation. 
Using our original criteria of credibility of knowledge within each evaluation 
design, credible knowledge can be classified in two categories, externally 
legitimated and locally legitimated. Again, there is a pattern seen within the 
six designs in terms of credible knowledge. Three approaches — positivism, 
critical theory, and aesthetics — are dependent upon a credible externally 
legitimated knowledge. Conversely, action research, interpretivism, and 
collaborative research designs value locally legitimated knowledge as credible. 



of power and credibility, the resulting table (see Figure 1 below) suggests the 
conditions under which each of the six evaluation approaches would be successful 



If we combine this analysis of the six designs along the dimensions 
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as a political strategy to fashion a dominant coalition* 



Insert Figure 1 here 

When there is a high power imbalance in favor of those in authority 
and an externally legitimated belief in science as a credible way to know 
something^ positivif!tic evaluation designs e.\ most likely to fashion a dominant 
coalition. Under similar conditions » except that the externally legitimated 
knowledge base involves the process and substance of critique, critical theory 
is the effective strategy to achieve a dominant coalition. A high power 
imbalance (towards the evaluatees) coupled with locally legitimated knowledge 
are the conditions under which action research is an effective political 
strategy. 

Under the conditions of a low power imbalance and an externally 
legitimated knowledge base that is personified in experts, aesthetic designs 
seem to be the appropriate mechanism by which to fashion a dominant coalition. 
Low power imbalance and a belief in locally legitimated knowledge indicate two 
appropriate designs. When the knowledge legitimated is that of the 
practitioner, collaborative s-esearch is the political strategy to fashion a 
dominant coalition. When the legitimated local knowledge is less based in 
practitioner expertise and more in a belief that what takes place "here" is more 
worthy than knowledge bases external to the evaluation situation, interpretivisra 
is the political strategy of choice. 

In the final analysis, however, it would be a mistake to consider the 
conditions and the resulting appropriate design as absolute. In practice, the 
choice of an evaluation design is recognized, we would argue, as a political 
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strategy and such a choice may alter the conditions themselves. We expect that 
politically successful (in terms of fashioning a dominant coalition) evaluations 
are iterative processes, a series of moves and countermoves that, in the end, 
produce a design or aeries of designs. This seems to require evaluators that 
are politically adept and methdologically flexible, 

CONCLUSIONS 

It is apparent from our analysis here that a sociology of knowledge 
approach exposes much of the tcken-for-^granted assumptions in evaluation 
research. Evaluators seem to take a number of things for granted. First, 
regardless of the design they employ, they take-for-granted that evaluation 
research does not have the characteristics of other social situations. Our 
analysis shows that this is not fully the case. Evaluation situations have the 
same bases in patterns of social relations and beliefs as any other social 
endeavor. Evaluation designs are not only political strategies, but they must 
be considered as political as any other social design, plan or program. Second 
and related, (specific) evaluators take for granted that the evaluatees will 
suspend a belief in the vested interests of che parties to the evaluation. That 
is, that the evaluatees will also accord evaluation a special status and treat 
it as an unusual social situation. Thirds many evaluators take for granted a 
"precontractual basis of solidarity" in an evaluation situation, Positivists 
assume it emerges from the legitimacy of science. Action researchers, 
collaborative researchers, critical theorists and asesthetic evaluators all also 
argue that trust is a precondition. Only the interpretivists, as is consistent 
with their approach, view trust as soraethinj? to be developed through the usual 
processes of social interaction. Fourth, evaluation researchers seew to take 
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for granted that the salient outcome of the evaluation situation is a factual 
report and/or a set of values upon which future actions can be based. Our 
analysis suggests that another salient outcoine is a new political arrangement 
between the parties to the evaluation situation. Further, we 'Jould argue that 
this may well be the primary basis of future action. Factu and values may be 
incidental in the political arrangement, but in thers^selves do not constitute a 
vehicle through which decisions will be made. 

It is also apparent that v must be careful about promoting 
evaluation in general as a standard policy in education. Certainly, programs 
should be evaluated, but to have a policy requiring evaluations as a periodic 
and usual process is to have a policy about how political arrangements will be 
negotiated. To specify the kinds of evaluation designs that are appropriate, 
furthermore, is to define a specific political role benefiting specific parties 
in the evaluation situation and in the educational system itself. To us, then, 
educators and evaluators would do well to consider the need for, the design of, 
and the political outcomes likely in any specific evaluation situation, and to 
eschew general policies about the necessity and form of evaluations. 

Finally, we would argue that evaluation designs need to be examined 
from a range of perspectives beyond that employed here* Only then will we 
understand the full social meaning of educational evaluation. 
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